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This publication examines research on the quality of teacher 
licensing examinations, explaining that state examinations do not necessarily 
guarantee that teachers have the breadth and depth of subject knowledge to 
teach all students to high standards. Researchers focused on English/language 
arts, mathematics, and science, analyzing actual exams published by 
Educational Testing Service (ETS) , which publishes the Praxis series, and 
National Evaluation Systems (NES) , which designs state-specific examinations. 
They investigated the approximate grade level of the test, how challenging 
the test questions were, and whether the knowledge was relevant to teaching. 
Results found that the ETS series of essay examinations, which requires 
candidates to demonstrate their depth of knowledge, is a good measure of 
teachers’ skills. However, this series is required by far fewer states than 
is the lower level multiple-choice examination. The NES Massachusetts 
literacy and communications skills examination asks more complex questions 
than do other examinations. Most tests examined were multiple choice 
assessments dominated by high-school-level material. There was no evidence of 
content at the baccalaureate level in any test. After presenting a summary of 
the tests, the article discusses who determines test content, reasons for a 
minimal approach to teacher licensing, and whether licensing examinations 
mean anything. (SM) 
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VER THE PAST SIX MONTHS, WE HAVE 
been asked on countless occasions why 
we are spending so much time on teacher 
quality issues these days. People seem 
surprised that an organization whose mission i^ 
squarely focused on closing the achievement ~~ 
gap separating poor and minority students from 
other young Americans would be pushing so 
hard for higher standards for teachers. “Set 
higher standards for teachers,” they say, “and 
minority kids will suffer because they will have 
fewer teachers who look like them.” 

Let us be clear 
from the beginning 
that we reject 
categorically the 
assertion that higher 
standards inevitably 
mean fewer minority 
teachers. Underneath 
that argument is a 
thinly veiled 
suggestion that people 
of color are somehow 
unable to meet high 
standards. Yet all of 
our experience 
suggests just the 
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opposite: minority and low-income students, 
including education students, can meet high 
standards if they are taught to high standards. 
The point is not to set standards below where 
they should be out of some misguided 
sympathy-or equally misguided belief that 
what minority kids need most is teachers who 
simply look like them. Rather, the point is to 
raise the quality and intensity of the education 
they receive. 
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To those who argue-for reasons of diversity 
or because of fears about supply-that the 
standards should be kept where they are, we 
make a simple suggestion. 

0 First, go spend time sitting in the back of 
classrooms, especially classrooms in high- 
poverty schools. Or join our staff as they work 
with teachers in those classrooms. You’ll see 
some stunningly good teachers, but you will 
also see teachers who quite obviously cannot 
get their students to state or local standards 
because they, themselves, don’t meet them. 

0 Then, once you have a feel for the problem, 
take a look at the growing body of research on 
teacher quality and student achievement, much 
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of it summarized in the Education Trust’s Good 
Teaching Matters or What Matters Most from the 
National Commission on Teaching and America’s 
Future. Both of these reports underscore a simple fact: 
teacher quality is the single most important factor in 
student achievement. 

Look, too, at who gets our weakest teachers: the 
very students who most need our best. 

In the end, if you spend as much 
time in high-poverty schools as we do, 
you can’t not care about teacher 
quality. Well-educated and well- 
supported teachers can help all children 
to soar to heights literally 
unimaginable to their poorly educated 
and poorly supported peers. 

Unfortunately, existing 
mechanisms are not even close to 
adequate for assuring teacher quality. 
Seven states have no licensing ' 
examinations for teachers. The remaining 44 (we 
. include D.C.) require examinations, but the 
combination of too-low content and too-low passing 
scores renders these systems effective in excluding 
only the weakest of the weak. 

This does not mean that all teachers are poorly 
prepared. On the contrary, both our own experiences 
and recent research prove that many teachers are 
wonderfully educated. Such teachers often tell us that 
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they are insulted by the low level of content exams. 

But the insults to children are even greater. Many 
are being shortchanged daily by poorly prepared 
teachers because we have failed to set high enough 
standards for entry into the field of teaching. 

Every American should be deeply concerned 
about the information in this report. But low-income 
families, as well as those of color, should be most 
worried of all, for the simple reason that the schools 
that serve them are the most likely to hire from the 
bottom of the pack. 

That we dramatically ratcheted up our standards 
for students without insisting on commensurate 
increases in standards for teachers is a chilling 
indictment of all of us: K-12 leaders, policymakers, 
higher educators and advocates. As they say, however, 
it is never too late. There are some things that states, 
districts and colleges can do immediately to reduce 
the problem! Other things will take more time. 

We must begin now-and we must focus on: 

° Rigorous preparation for intending teachers; 

° Higher standards for entry into the field of 

teaching-including tough academic examinations; 
and 

° Ongoing support for current teachers. 

If we are relentless about these three things, no 
excuses and no exceptions, our children will succeed. 

Kati Haycock 
Director 



IS CONTENT SUFFICIENT? 

In both our analyses and our recommendations, we have concerned ourselves primarily with content 
knowledge. We have done so not because we believe that deep content knowledge is sufficient or because we 
think other things are unimportant-things like content pedagogy, knowledge of how children learn, and belief 
systems, among others. (Nobody who does as much work as we do in higher education could possibly 
believe that deep subject matter knowledge always equals good teaching.) Nor, it may be important to point 
out, do we believe that content knowledge is forever fixed when teachers complete their preparation. 

Rather, we believe that the grasp of the core concepts and structure of a discipline with which one exits 
from college is a critical foundation for teaching: if that foundation is inadequate, no instructional wizardry 
can make up for it. Moreover, though there are many voices within the teacher education reform community 
for the importance of pedagogy and the like, there are few such voices raised around content, even though 
teachers who are furthest ahead in implementing their state’s standards often struggle more with content than 
anything else. '• 

So, for the time being at least, we’ll be a bit shrill. Until, that is, more faculty in the Arts and Sciences 
join in and there is, finally, a balance in the conversation and the reform effort. 
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How Teacher Licensing Tests 

Fall Short 

by Ruth Mitchell, Ph.D., aud Patte Barth 
The Education Trust 



ETTlEACHERS hold a position of public trust. 

The community^ for its part, is responsible for 
supporting the goals of education and for 
allocating sufficient funds to get the job done. But 
ultimately, whether or not our children succeed 
academically depends on the knowledge, skill and 
commitment of their teachers. 

With so much riding on the 
quality of teachers, the public 
needs assurance that every 
student is taught by 
professionals who know a lot 
about the subjects they teach. 

This is especially true for 
students who lack resources at 
home. These students-much 
more than their more advantaged 
peers-depend almost exclusively 

on their teachers for academic 

content knowledge. Most states 
administer teacher licensing examinations as a kind of 
guarantee that teachers know enough about their 
subjects. But do these tests really certify that teachers 
have the breadth and depth of subject knowledge to 
teach all students to high standards? 

The short answer is no. Over the last year, we 
examined the tests most commonly used for licensing 
beginning teachers. In general, state licensing 
requirements place more emphasis on prospective 
teachers’ pedagogical knowledge than on their content 
knowledge. Moreover, the subject area tests we 
examined are too weak to guarantee that teachers have 
the content they need to teach students to high 
standards. 

0 SECONDARY EDUCATION: Whereas 44 
states require candidates for secondary licenses to take 
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some kind of licensing examination, only 29 require 
them to take tests in the subject area they will teach. 
The content in the subject tests, with a few (under- 
used) exceptions, is within easy reach of many of the 
students the test-takers are expected to teach-about 
the same as in high-level high school courses. 

0 ELEMENTARY 

EDUCATION: Seven 
states have no examination 
requirements for candidates for 
elementary certification. The 
remainder require examinations 
that cover pedagogy and 
rudimentary general knowledge 
and skills. In general, these tests 
assess verbal and mathematical 
literacy at about the tenth grade 
level. 1 

As Lynn Steen, a national 
adviser to our study, put it: “Why 
should prospective teachers go to college if this is all 
they need to know?” 

The long answer to our question about the 
adequacy of existing licensure examinations is a 
complicated tale that has its origins in good intentions, 
but in the end pits students’ needs against institutional 
interests and adults’ right to jobs. At its core, the 
system is designed to prevent false negative 
judgments (about either candidates or the institutions 
that produce them). But if we were truly concerned 
about students, we would be more worried about the 
false positives. 

Like many practices in education, the criteria for 
teacher licensure were established in an era that held 
modest academic expectations for the majority of 
young people. In the last decade, however, K- 12 
education began a transformation: high academic 
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standards are now the expectation for all, not some 
students. But while we raised standards for students, 
we have yet to make corresponding increases in 
standards for teaching. 

Unfortunately, raising the level of teacher 
licensing examinations is no simple matter. The 
process for defining both test content and what 
constitutes “passing” takes many factors into account 
that compete directly with the goal of certifying that 
candidates have a strong command of subject matter. 
Projections of teacher supply and demand, protection 
for the state and university against legal challenges by 
unsuccessful candidates, and the authority of 
universities are all considered in the licensing 
equation. 

State licensure policies also rest on assumptions 
about matters that do not enter the licensing equation 
but should. Many assume, for example, that the act of 
majoring or passing a certain number 
of courses in accredited universities 
in itself certifies a sufficient level of 
content knowledge. Licensure policies 
also typically assume that what 
beginning teachers don’t know now 
they will learn in time. Underlying 
the whole process is the assumption 
that teachers only need to know the 
content that is expected of their 
students, and maybe just a little bit 
more. For all these reasons, licensure 
examinations don’t contain as much 
content as we believe fully qualified 
teachers need in order to educate all students to high 
levels of understanding. 

Another wrinkle in the process is the 
establishment of passing scores-the cut-off point 
between passing and failing the licensing examination. 
Passing scores are not set by the test publisher; rather 
they are established state by state. In some states, 
candidates can pass subject-matter exams by correctly 
answering as few as half the test items. In areas of 
short supply, states may still require candidates to take 
the test, but will waive the requirement for minimum 
performance. In such cases, any old score will do. Far 
from a guarantee of high professional standards, 
certification requirements often define teaching down, 
even while public demands for teacher performance 
are being ratcheted up. 



There are no bad guys to blame for this situation. 
Officials who make certification policies are pushed to 
balance teacher shortages, growing student 
enrollments, the demand to reduce class size AND 
make sure a teacher is in every classroom. The 
colleges that prepare teachers struggle to respond to 
what often seem to be conflicting state mandates and 
confusing messages from school districts about what 
is important. At the same time, universities want to 
protect both the academic freedom of professors and 
the right of students to choose how to fulfill core 
academic requirements. Up and down the line there is 
concern about the potential impact on the quantity and 
diversity of the teacher force if the bar is raised high. 
The pressures on state policymakers are to minimize. 
In the end, test publishers respond by giving their 
customers what they say they want: a reliable method 
for measuring the lowest possible teaching 

competencies that can hold up in 
court. 

The institutional and logistical 
issues that influence the makeup of 
teacher exams are real. Yet they have 
been allowed to overshadow what 
should be the paramount 
consideration of teacher certification: 
Can this individual teach all students 
to high, not minimum, standards? 

We looked at one aspect of this 
question, content knowledge, because 
we find it is the most neglected in 
teacher education reform. In addition, 
although it is by no means the only characteristic of a 
good teacher, we believe content knowledge is the 
most central. Without it, no manner of teaching skill 
can possibly yield high student achievement. 

WHAT WE LOOKED FOR 

Because we were interested in how much 
teachers know about their subjects, we looked only at 
examinations of subject matter and general 
knowledge; we did not examine tests of pedagogical 
skills and knowledge. There are too many content 
examinations for the scope of this study. We therefore 
limited our study to English/language arts, 
mathematics and science. Within these subjects, in 



States Not Requiring 
Licensing Exams: 98-99 

Idaho 

Iowa 

_ _ NorthJQakata 

South Dakota 

Utah 

Vermont 

Washington 

Source: NASDTEC Manual on the 
Preparation & Certification of 
Educational Personnel 98-99, Table 
G-4 






Tlhe Educatkm Trust 



TMmMImg K-B& 



C'OMOOCniNG THE STUDY 

Choosing particular tests to investigate was relatively straightforward once we limited the subject 
areas. There are only two major providers of teacher examinations: Educational Testing Service (ETS) 
which publishes the Praxis series and National Evaluation Systems (NES) which designs state-specific 
examinations. A larger number of states require Praxis, but a larger number of students take NES 
examinations, since NES contracts with the big-population states such as Texas and New York. ETS 
and NES exams are taken by the vast majority of prospective teachers. We therefore based our 
analysis on tests published by these two providers. 

As much as possible we analyzed actual exams as opposed to their widely available sample forms. 
ETS granted us controlled access to examine their Praxis series examinations on two occasions. 
Because NES contracts with individual states, access to their tests is in the hands of the appropriate 
state agency. Our conclusions are based on complete examinations from a single NES state and study 
guides from six others. 

The content analysis of these examinations was conducted initially by a team comprised of 
Education Trust staff, represented by the authors, and outside consultants. In this method, the team 
worked through the tests as if they were teacher candidates, noting relevant information about each 
item. The methodology and preliminary judgments were validated by a distinguished national review 
panel who went through the analytical model in an abbreviated form. The national panel has endorsed 
the judgments reported here. 



NATIONAL REVIEW PANEL 

<Saa 5 SutrirSGB, past president of the National Council of Teachers of Mathematics (NCTM), now Senior 
Program Officer at the National Academy of Sciences (NAS) and associate researcher at the University 
of Wisconsin, Madison; 

Gsorg® RfOBBB©tr, senior lecturer emeritus of chemistry at the University of California-lrvine; 

Daira Jooues, Dean of the College of Liberal Arts, Towson University, Maryland; 

Lymm Airitouir Steemi, past president of the Mathematical Association of America (MAA) and professor at 
St. Olaf College, Northfield, Minnesota; 

istuigsmiE® Scot!, executive director of the National Center for Science Education; and 
Geoirg® PulHImairi), associate professor of English, Georgia State University 

OOTSroE CONSULTANTS 
Erma An)<d]®r§©m), science education consultant; 

liradifeirsS FopsEeOO, Program Officer, Center for Science, Mathematics, and Engineering Education at the 
National Research Council; and 

Amy Kuntosa StempaO, high school consultant with the Edison Project. 
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WHAT IS SIMPLE <CM COMPLEX? 



Test items were classified as either “simple,” “moderate,” or “complex.” These categories refer to the level of 
sophistication demanded by the test question. Here are some examples (answers appear in boldface type). 



SIMPLE 

A “simple” problem requires the simple recall 
or recognition of factual material. In math and 
science, the problem can be solved in one step or 
by applying a simple procedure. Simple items can 
be answered successfully with only a superficial 
level of understanding the concept. 

An example of a “simple” question: 

Which of the sales commissions shown 

below is the greatest? 6 

a) 1% of $1,000 

b) 10% of $200 

c) 12.5% of $100 

d) 15% of $100 

e) 25% of $40 

This problem requires the application of one 
simple arithmetic procedure to answer 
successfully. 

COMPLEX 

A “complex” item is a multi-step problem that 
also requires the development of a strategy 
drawing on more than one domain. Open-ended 
items, such as essays and mathematical proofs, 
tend towards more complexity, but we occasionally 
found complex multiple-choice items. Complex 
items require a deeper understanding of concepts 
than either simple or moderate questions. 

Example of a “complex” problem: 

Use the graph below to answer the question that 
follows: 7 * 




The graph represents a system of linear 
equations. For what values of m will the solution to 
the system be in the first quadrant? 

a) m < 1/2 

b) m < 5 

c) m> 1/2 
d) m> 5 

Gail Burrill wrote that this Algebra 1 question is 
complex in that it requires students “to use 
understanding of slope, graphs and solutions to 
find the answer and gets at more than just 
applying procedure.” 

MODERATE 

“Moderate,” not surprisingly, is between simple 
and complex: the item requires more than one 
step, but not necessarily a strategy drawing on 
other domains. 

An example of a “moderate” question: 

Directions for Questions 5-6: the group of 
questions below consists of four lettered headings 
followed by a list of phrases or sentences. For 
each sentence, select the one heading that is 
most closely related to it. One heading may be 
used once, more than once, or not at all. 8 

a) Nephrons 

b) Flame cells 

c) Malpighian tubules 

d) Skin gills 

5. Function in both arachnids and insects C 

6. Have cilia to guide waste products to excretory 
pores B 

This question asks for the recall of information, but 
it also requires the test-taker to use that 
information to classify the items, making this a 
moderately complex question. 



While we hoped to find complexity, "simple recall” items were the most commonly seen types of questions in 
the tests we examined. 



Thee Edkacatioini Tmnst 



TMnalktag DS=M) 



order to be as fair as possible, we devoted most of our 
attention to the highest level tests currently used. 

The study was guided by the following questions: 
° What is the approximate grade level of this test? 
We wanted to gauge when the content covered on the 
test is normally taught and learned. Our grade level 
designation is a judgment of the test as a whole. It 
represents the grade in school or the year in college at 
which a typical student would have learned enough to 
answer most of the test questions correctly. 

Our assignment of “grade level” does not take 
into account passing scores. However, it’s important 
to note again that teachers can become licensed in 
some states by correctly answering as few as 45% of 
the test items. While it’s not possible to say exactly 
how such a low passing score affects the designation 
of grade level, it is likely to reduce the test’s effective 
difficulty level substantially. 

° How challenging are the test questions? 

We evaluated the degree of sophistication demanded 
by each test item. “Simple” items required only one 
step and a simple procedure, for example, the simple 
recall of factual information. On the high end were 
“complex” problems that required multi-step strategies 
involving more than one domain, for example, a math 
problem that draws on concepts from both algebra and 
geometry. (See “What is Simple or Complex?”, page 6) 
° Is this knowledge relevant to teaching? 

While we assumed that teachers should know a great 
deal more than their students, we also wanted to see if 
the content is connected to what they will be expected 
to teach. For example, linear algebra is typically not 
encountered until college. However, the depth of 
understanding that linear algebra can develop is 
relevant to teaching algebra in middle and high 
school, and probably to algebraic concepts at the 
elementary level as well. 

Our team of analysts went through each test item 
by item, answering each question as if we were 
teacher candidates ourselves. Items were classified 
according to the dimensions listed above. The 
conclusions summarized in this report are based on 
the initial documentation by the team and validation 
from our national panel of advisers . (see “Conducting 
the Study,” page 5). 



WHAT WE FOUND 



We found a few things to admire, a lot of 
disappointment and one huge gaping hole. One bright 
spot was the series of “essay” examinations published 
by ETS, which required candidates to demonstrate 
their depth of knowledge. The essays tended to cover 
more sophisticated content, although not quite at the 
level of a B.A. On a discouraging note, the essays are 
required by far fewer states than the lower level 
multiple-choice versions. 3 

We were also impressed by the sample items for 
the Massachusetts literacy and communications skills 
exam published by NES. These questions, in the 
words of Dan Jones on our advisory panel, were “of a 
higher degree of complexity and expectation than any 
of the others 
we looked at.” 

States that 
contract with 
NES define the 
content and 
level of the 
assessments 
and there is 
considerable 
variation 
among NES 
examinations. 

Because we did not have access to the complete 
Massachusetts exam, we cannot make statements 
about its overall quality, particularly since the one 
NES-published test we reviewed did not reflect the 
same complexity as the Massachusetts sample items. 
But what we were able to see showed considerable 
promise. 

In contrast to these few bright spots, the majority 
of tests we examined were multiple-choice 
assessments dominated by high-school level material. 
A few, notably in science, devoted a significant 
proportion of questions to content learned in middle 
school. Dan Jones found many of the English 
language arts questions “disappointing,” saying that 
these tests offer “empty generalizations as the right 
answers.” 

According to our consultants and reviewers, most 
of the tests we examined could be easily handled by 
advanced high school students. Lynn Steen asserted 
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that the math tests “could be passed by a B+ student 
upon leaving high school.” When one factors in the 
low passing scores in some states, passing a licensing 
exam can mean nothing more than a high school 
diploma. 

We found no evidence of content at the 
baccalaureate level. Although a bachelor’s degree in 
itself may not certify that the content is relevant to 
teaching a K-12 curriculum, we did expect to see 
content demanding a level of sophistication acquired 
through four years in college. Not one test was up to 
the level of a graduating college senior. 

More to the point, we did not find the content that 
our panel believes is essential for teachers charged 
with getting all students to high standards. This 
“knowledge for teaching” involves the deep mastery 
of an academic subject that goes beyond, but is still 
connected to, the level of highest student achievement 
in K-12. It also equips teachers to answer the 
perennial question from students-“Why are we 
learning this?”-with reasons based on understanding 
of the discipline, rather than “Because you will need it 
in the next course.” Many educators are familiar with 
the term “pedagogical content knowledge” which 
includes being able to find cognitive bridges such as 
metaphors, pictures, or manipulatives that enable 
students to understand concepts. 4 For example, the 
Praxis II Mathematics: Pedagogy test asks candidates 
to write on this question: 

A small group of students in your seventh-grade math 
class is unable to determine whether two fractions are 
equivalent. Describe a strategy, using pictures or 
manipulatives, that you could use to help foster the 
students' conceptual understanding of equivalent 
fractions. Your strategy should stress understanding 
of what it means for fractions to be equivalent and the 
development of the ability to determine whether 
fractions are equivalent . 5 

What this question does not require, and 
mathematics pedagogical content knowledge as such 
does not encompass, is the understanding of 
equivalence as an essential component of 
mathematical thinking. A seventh-grade teacher 
should be able to inspire students by referring to 
equivalence as a technique in sophisticated proofs. 
This is the understanding that we are calling 



“knowledge for teaching.” In other words, middle- 
school teachers need to know how seventh-grade math 
is foundational to very sophisticated mathematical 
concepts. 

All teachers, including elementary teachers, need 
to understand not only the structure of the academic 
discipline, but how, by organizing knowledge in 
specific forms, it contributes to understanding of the 
world. They should know why our civilization values 
the knowledge they are imparting to students, so that 
they can convey some of the passion for beauty and 
order that their discipline embodies. 

Knowledge for teaching is a gaping hole in 
licensing examinations. For this reason alone, we 
cannot say that any of these tests satisfies our first 
question: Do these tests certify that teachers have 
sufficient subject knowledge to teach all students to 
high standards? 

While none of these tests adequately addresses 
content, some of them were found to be better than 
others. A summary of the tests follows. 

ELEMENTARY EDUCATION AND 
BASIC LITERACY 

It is difficult to evaluate the content knowledge in 
elementary licensing examinations. The tests that 
elementary teachers most commonly take are 
concerned largely with pedagogy, not subject matter 
knowledge, and therefore lie outside this study. 
However, many states require a test of basic literacy 
for all prospective teachers, which by default becomes 
the “content” test for elementary teachers. 

Both ETS and NES provide basic literacy or 
general knowledge tests. Overall, these tests were 
characterized by simple recognition or recall of 
general subject matter. A typical treatment of content 
is seen in this literature question: 

In the meanwhile there came along a single red ant on 
the hillside of this valley, evidently full of excitement, 
who either had dispatched his foe, or had not yet taken 
part in the battle; probably the latter, for he had lost 
none of his limbs ; whose mother had charged him to 
return with his shield or upon it. Or perchance he was 
some Achilles, who had nourished his wrath apart, 
and had now come to avenge or rescue his Patroclus. 
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The [preceding] passage makes use of analogies that 
originate in 

a) Roman mythology 

b) Elizabethan drama 

c) Greek epic 

d) the New Testament 

e) Arthurian legends 9 

In questions like this one, the test-taker either 
knows the answer or does not. It reveals nothing about 
the candidate’s ability to interpret, analyze or 
otherwise make use of this knowledge. 

ETS publishes the widely used Praxis I, also 
called the Pre-Professional Skills Test (PPST), which 
is a test of basic skills or literacy. NES publishes 
similar exams, including CBEST for the state of 
California. These literacy exams are intended as 
qualifying tests for entry into teacher preparation 
programs and are designed to be administered around 
the second year of college. However, in many states 
these tests can be taken at any point before licensure 
and many prospective teachers take them after 
completing their formal training. In these states, the 
literacy test becomes by default a qualifying 
examination for teaching. Indeed, in some states it is 
the only content test elementary teachers take. 

Praxis I addresses only reading, writing and 
mathematics. None of these sections exceeded high 
school level, and at least two-thirds of the 
mathematics items were judged to be middle school. 
An analysis comparing the distribution of Praxis I 
math items to the 1996 National Assessment for 
Education Progress (NAEP) for mathematics (see 
chart below) seems to indicate that NAEP emphasizes 
a better balance of mathematics, even at the eighth 
grade level, than does Praxis I. 



Praxis I 


NAEP 
Grade 8 


NAEP 
Grade 12 


Number 37.5% 


25% 


20% 


Measurement 5% 


15% 


15% 


Geometry 15% 


20% 


20% 


Data Analysis 20% 


15% 


20% 


Algebra 125% 


25% 


25% 





The PPST reading passages were on the level of 
the National Geographic , typically high school 
readings but clearly accessible to middle and upper- 
elementary school students. In general, the questions 
that referred to these passages asked for either direct 
recall of information in the passage or for obvious 
inferences or interpretation. 

Alice Fletcher, the Margaret Mead of her day, assisted 
several American Indian nations that were threatened 
with removal from their land to the Indian Territory. 
She helped them in petitioning Congress for legal titles 
to their farms. When no response came from 
Washington, she went there herself to present their 
case. 

According to the statement above, Alice Fletcher 
attempted to: 

a) imitate the studies of Margaret Mead 

b) obtain property rights for American Indians 

c) protect the integrity of the Indian Territory 

d) become a member of the United States Congress 

e) persuade Washington to expand the Indian 
Territory 10 

This question is simple on several levels. The 
reading passage itself is straightforward with 
relatively simple vocabulary and syntax. The question 
asks only for a literal recognition of information 
provided in the passage. This question might easily be 
found on a middle-school reading exam. 

The writing questions, or prompts, in the paper- 
and-pencil version were bland and general with no 
specified audience or discernible purpose. An 
example: “Which of your possessions would be the 
most difficult for you to give up or lose?”" George 
Pullman of the national advisory panel described the 
writing prompts this way: “The weakness tends to be 
acontextuality - the kind of ‘once off the top of your 
head to no one in particular for no reason except to 
test your writing’ test that is so common. The better 
prompts have a clear context and a set audience and 
purpose.” 

The basic literacy exams showed little 
complexity; rather the test items tended to require 
only simple recall or the application of a set 
procedure. These tests are taken by students at any 
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point between two and four years into their college 
careers. Yet overall we found these tests to be far less 
difficult than either the SAT or ACT - tests that 
students should have performed on with some success 
in order to have been admitted to their universities at 
the outset. These tests were mostly at the eighth to 
tenth (sometimes seventh) grade level. 

SECONDARY MATHEMATICS 

Most of the content of the Praxis II and NES 
mathematics examinations can be found in a broad 
high school curriculum. Only a few questions went 
beyond calculus or addressed concepts typically not 
learned until the first two years of college. Most of the 
items on the exams differed little from what can be 
found on a test of high school mathematics. 

A notable feature of most of the mathematics tests 
was that mathematical definitions and basic formulas 
were provided up front (e.g., formulas for the area of 
a triangle and the circumference of a circle). Even 
though we tended to view recall items as low level in 
our analysis, this was one instance when our 
reviewers believed the ability to recall was important. 
Lynn Steen commented that by not asking candidates 
to produce formulas on demand, it’s as if the test 
publishers “don’t care whether candidates actually 
know anything, but only whether they can carry out 
learned procedures.” 

Of the Praxis II tests, we looked at three which 
contained the most advanced material. These were 
Mathematics: Content Knowledge (0061); 
Mathematics: Proofs, Models, and Problems, Part I 
and Part II (0063 and 0064). The following table 
summarizes the content distribution of Mathematics: 
Content Knowledge, which is a 50-item multiple 
choice test: 



Content distribution by NAEP categories 



Praxis II 
Math (0061) 


NAEP 
Grade 8 


NAEP 
Grade 12 


Number 10% 


25% 


20% 


Measurement 0% 


15% 


15% 


Geometry 34% 


^0%~ 


2T5% 


Data Analysis 10% 


15% 


20% 


Algebra 46% 


25% 


25% 





This distribution should be compared to the table 
on page 9 for the Praxis I mathematics. Despite the 
obviously larger percentage of algebra items, our 
analysis concluded that only eight of 50 items were 
clearly college level. In addition, 70% of the items 
were simple, and only 1 6% complex. 

Although the vast majority of the items did not 
require complex problemsolving, more than half of 
the questions asked for some application of concepts 
to problemsolving situations. Steen thought these 
multiple-choice tests contained “a significant number 
of unusual questions that would exercise the 
metacognitive capabilities of candidates.” 

In contrast, he thought the Mathematics: Proofs, 
Models, and Problems (0063 and 0064) “included 
nothing that was not absolutely straightforward.” We 
considered these examinations together because they 
consist of only four and three problems respectively. 
We saw two forms (different years of the same test) 
for a total of 14 problems, ranging from geometry to 
linear algebra. However, only four out of the 14 
problems were concerned with topics taught in 
college, and just four were considered complex. 

Mathematics items in NES tests vary from the 
mostly routine questions in the complete state test we 
reviewed to the more complex and sophisticated item 
from the Massachusetts test cited on page 6. The 
complete secondary mathematics test we reviewed had 
a large number of items placed in a real-world 
context, which at first sight looks like a good idea. 

But frequently the contexts are thinly veiled 
procedural or even recall items. Many of the contexts 
are contrived, a few are silly, and at least two are 
wrong. Some items conceal a tiny mathematical topic, 
reached after wading through a heavy context. 

This math test could be answered with ease by a 
mathematics student after finishing AP calculus, or 
even before, since the only calculus items are often 
included in the pre-calculus course. 

In general, the tests assess mostly tenth to 
eleventh grade level content. Nothing in either the 
Praxis II tests or the NES tests probes the intellectual 
substance of college mathematics that should equip 
high school teachers with robust backgrounds for 
dealing with the myriad ideas that will emerge from 
discussion with students. 
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Bn the Praxis II series, we had to choose among a 
large variety of tests in several scientific disciplines. 
As in the case of mathematics, we wanted to look at 
the tests with the highest academic challenge. But we 
also had to choose those most used by states buying 
Praxis II tests (for example, only 2 1 candidates 
throughout the entire country took the Physical 
Science: Content Essays [0482] test between 1995 and 
1998). The three most used Praxis II tests in science 
that are also the most challenging are General 
Science: Content Essays (0433), Biology: Content 
Knowledge, Part I and Part II (0231 and 0232). 

Like the other Praxis II essay examinations, 
General Science: Content Essays has only three 
questions, which candidates answer in writing. 
Although three questions cannot measure the breadth 
of knowledge required at the secondary teaching level, 
the examination assesses the candidate’s ability to use 
and analyze critical concepts in science covered in 
introductory college courses in life science, physics, 
chemistry, and earth science. The three questions 
broke into six parts: the most challenging were the 
physical science and general science questions; of the 
life science questions, one required procedural 
knowledge and the other only tapped recall. Three of 
the parts required complex, multi-step answers, but 
the life science question that required recall also only 
asked for a single-step answer, surprising in a test 
requiring written responses. 

This example is typical of the science content 
essays in that the challenge ranges from recall to 
complexity: 




The potometer shown above is used to estimate 
transpiration rate in plants. 
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A. Define the process of transpiration. 

B. Identify three variables that would affect 
transpiration rate. Describe how the potometer can 
be used to test these variables. 

C. Discuss how and why a change in each variable is 

12 

expected to affect the transpiration rate. 

The Biology: Content Knowledge, Part I (0231) 
test, with 75 multiple-choice items, tested knowledge 
of the basic principles of science; molecular and 
cellular biology; classical genetics and evolution; 
diversity of plants and animals; ecology; science, 
technology, and society. Almost half of its items 
could be answered with simple recall of information; 
only seven required multi-step problemsolving, and 
only 12 required the application of a concept. 

The second part of the Biology: Content 
Knowledge, Part II (0232) test also has 75 multiple- 
choice items, but they test knowledge that would be 
acquired in high school advanced or honors classes 
and some in first-year college biology classes. The 
questions covered molecular cellular biology; classical 
genetics and evolution; diversity in plants and 
animals; and ecology. Compared with Part I, there 
was a considerable increase in the number of items 
requiring either moderate or complex problemsolving 
(57 in Part II), and in procedural or conceptual 
knowledge (62). 

While all three of the tests examine knowledge of 
the “big ideas,” the concepts defined in the National 
Science Education Standards and many state 
frameworks for science, only Biology: Content 
Knowledge Part II reflects the depth of knowledge 
that one would expect of a beginning teacher of 
biology. Biology: Content Knowledge Part I does not 
cover a wide or deep enough selection of topics to 
adequately reflect what a high school science teacher 
needs, even as a beginner. 

The NES science tests, like the mathematics tests, 
were extremely variable. The complete test we 
examined had tests in general science, biological 
science, chemistry, and physics. A feature of NES 
examinations which differentiates them from ETS’s 
Praxis series is that NES tests include pedagogical 
questions within the test. In the general science test, 
11% of the questions were concerned with the 
pedagogy of science; in the physics test, 15%; the 
biological science test, 17%; and 20% in chemistry. 
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In no case did the academic knowledge tested go 
beyond grade 12 level, and a large number of the 
items in all four tests were at the grade 8, 9, or 10 
level (even some at grade 7). In general, the NES 
complete test reflected the techniques of multiple- 
choice tests from 30 years ago. 

On a positive note, the NES-published 
Massachusetts sample test again provided an example 
of greater sophistication, asking for a written response 
to an earth science problem on earthquakes. Eugenie 
Scott and George Miller wrote that this problem 
“requires both scientific and engineering/social 
knowledge to answer all the parts ... If the objective of 
the sample is to have candidates prepare for science, 
technology and society (STS)-type items, then this 
should be effective.” However, without access to the 
whole exam, it’s not possible to know if the test 
adequately addresses breadth as well as candidates’ 
depth of knowledge of key concepts. 

SECONDARY ENGLISH/ 

LANGUAGE ARTS 

Secondary English/Language Arts tests are 
concerned with general knowledge about literature; 
knowledge of grammar, style, and etymology; and 
knowledge of resources (dictionaries, thesaurus, etc.). 
We looked at two Praxis II tests in English: English 
Language, Literature, and Composition: Content 
Knowledge (0041) and English Language, Literature, 
and Composition: Essays (0042). 

We examined two forms of the Content 
Knowledge test, published about five years apart. The 
test asked students to answer 150 items in two hours. 
The questions were mostly recall items to which a 
student either knows the answer or doesn’t. For 
example: 

If we could only get one of them new-fangled carriages 

that make no noise that Father O \ Rourke told him 

about, them with the rheumatic wheels. 13 

The underlined word in the lines above is an example of 

a) a metaphor 

b) a malapropism 

c) an oxymoron 

d) a synecdoche 



The first 65 or so items are about literature. The 
range is not wide: most of the items are concerned 
with classic English literature, nineteenth century 
American literature, and a smattering of world 
literature, the stuff of college survey courses. The 
other questions (the majority) ask about language use, 
editing, etymology, and research resources 
(dictionaries, thesaurus, etc.) 

The test was overwhelmingly concerned with 
breadth of content knowledge. The result was a 
superficial treatment-who wrote what, or where do 
you find this. Although taught in college survey 
courses or in English education courses (where 
technical grammatical content is often taught), the 
material is not conceptually too difficult for high 
school students. There are no questions demanding a 
depth of knowledge that would enable “people to 
show that they know how to do useful things with 
what they know,” in George Pullman’s words. 

The story was surprisingly different with the 
English Language, Literature, and Composition: 

Essays test (0042). The test is two hours long and 
requires two essays. One provides two poems, usually 
on the same theme, and asks for a close critical 
comparison. The second essay presents an issue in the 
English profession: one example asks the candidate to 
write on the canon and discuss attempts to expand it; 
another asks the candidate to compare two schools of 
criticism, practical and response criticism, for 
example. The scoring guides show that a high level of 
argument, knowledge of the subject, and writing 
ability is required to receive the highest score. 

The essay test demands the knowledge of the 
field and experience with close reading that is 
typically acquired in introductory courses to the 
English major. It comes closer to knowledge for 
teaching than other tests, but its questions are not 
constructed with the specificity that would require 
knowledge at the level of a college graduate. We 
therefore put it within the scope of a major who has 
completed the junior year. Even so, it remains the 
highest level achieved by any of the tests we 
examined. 

Only nine states require the Essays test (0042). 
Sadly more than 30 states certify secondary English 
teachers without testing any writing beyond the 
content-less essay in the basic skills tests. 
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The NES single-subject tests combined pedagogical 
and content material, but this time, within the same 
question. Here is an example from the ExCET 
(Texas) Preparation Manual: 

Write an essay in which you: 

° Analyze how the speaker uses metaphor to reflect on 
truth and ignorance, and 

° Propose learning activities for a specific secondary 
grade level that would help students build an 
understanding of the poem and explain why you would 
use these activities. 

[The poem is “ Truth ” by Gwendolyn Brooks] 

In NES English/language arts secondary tests, 
most of the items concerned literature; grammar and 
style; and pedagogy. Most of the answers to the 
literature questions would be found in college survey 
courses, because they include examples from the best 
known Greek tragedies and traditional British and 
American literature. Comprehension questions 
include passages from American Indian and African 
American writers, but are all on the level of simple 
recognition. 

Some NES states require written essays, usually 
providing a challenge on the level of a first or second- 
year college student. A Colorado prompt asks students 
to compare excerpts from the Rig Veda and a Navajo 
myth, and then discuss the functions of myth with 
analysis of the themes and stylistic devices of the two 
passages. While a college-level challenge, this prompt 
does not draw on the specific knowledge of issues in 
literary criticism to be expected of an upper division 
college student, as does the essay question cited from 
the Praxis II test, English Language, Literature and 
Composition: Essays (0042). 

THE MATTER OF PASSING SCORES 

Our judgments about the licensing exams are 
based on a best case scenario. In estimating the test 
level, we assumed that all items had been answered 
correctly. In the real world, of course, there are 
passing scores, which establish the cut-off point 
between pass and fail. Passing scores are set by the 
individual states and can vary considerably for the 
same test. For example, on the Praxis II test, 
Mathematics: Content Knowledge (0061) - a test we 



Praxis I Math — Passing Scores By State 1998-99 



estimated to be at the advanced high-school level- 
required scores range from Oregon’s high with a 
scaled score of 147 to Georgia’s low at 124. At the 
high end, an Oregonian need only answer about 65% 
of the questions correctly to begin teaching high- 
school mathematics. 14 In Georgia, a prospective 
mathematics teacher can become licensed by correctly 
answering fewer than half (about 46%) of the test 
items. Ironically, students would receive “Fs” for 
producing such scores in the classroom, yet this is all 
states require of their teachers. 

Passing scores for other teacher licensing tests 
show similar patterns of variation among states as 
well as a tendency for dumbing down. Nowhere is this 
trend more apparent than in the widely used Praxis I, 



State 


Passing 

Score 


Estimated % 
Correct To 
Pass 


% of Test 
Takers 

Nationally Who 
Would Fail to 
Make this Cut 


Virginia 


178 


68-73% 


44 


Hawaii 


176 


60-65% 


36 


Oregon 


175 


60-65% 


31 


Florida 


175 


60-65% 


31 


Kansas 


174 


60-65% 


26 


DC 


174 


60-65% 


26 


Delaware 


174 


60-65% 


26 


Alaska 


173 


53-58% 


22 


Wisconsin 


173 


53-58% 


22 


North Carolina 


173 


53-58% 


22 


Kentucky 


173 


53-58% 


22 


Georgia 


173 


53-58% 


22 


West Virginia 


172 


53-58% 


18 


New Hampshire 


172 


53-58% 


18 


Maine 


172 


53-58% 


18" 


South Carolina 


172 


53-58% 


18 


Arkansas 


171* 


53-58% 


15 


Oklahoma 


171 


53-58% 


15 


Nebraska 


171 


53-58% 


15 


Texas 


171 


53-58% 


15 


Nevada 


170 


45-50% 


12 


Montana 


170 


45-50% 


12 


Tennessee 


169 


45-50% 


10 


Mississippi 


169 


45-50% 


10 


Minnesota 


169 


45-50% 


10 ! 


* Effective July 1999 
See Sources Page 23 
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Praxis I Reading — Passing Scores By State 1998-99 



State 


Passing 

Score 


Estimated % 
Correct To 
Pass 


% of Test 
Takers 

Nationally Who 
Would Fail to 
Make this Cut 


Virginia 


178 


71-76% 


43 


North Carolina 


176 


63-68% 


30 


Alaska 


175 


63-68% 


24 


Delaware 


175 


63-68% 


24 


Hawaii 


175 


63-68% 


24 


South Carolina 


175 


63-68% 


24 


Wisconsin 


175 


63-68% 


24 


New Hampshire 


174 


63-68% 


19 


Oregon 


174 


63-68% 


19 


Kansas 


173 


55-61% 


16 


Kentucky 


173 


55-61% 


16 


Maine 


173 


55-61% 


16 


Minnesota 


173 


55-61% 


16 


Oklahoma 


173 


55-61% 


16 


DC 


172 


55-61% 


12 1 


Florida 


172 


55-61% 


12 


Georgia 


172 


55-61% 


12 


Nevada 


172 


55-61% 


12 


West Virginia 


172 


55-61% 


12 


Arkansas 


172* 


55-61% 


12 


Texas 


172 


55-61% 


12 


Mississippi 


170 


47-53% 


8 


Montana 


170 


47-53% 


8 


Nebraska 


170 


47-53% 


8 


Tennessee 


169 


47-53% 


6 


*Effective July 1999 
See Sources Page 23 



a basic literacy test that we judged to be at about the 
tenth to eleventh grade level. In most of the 25 states 
using this test, an elementary teacher can become 
certified by correctly answering somewhere between 
47 and 61% of the reading and mathematics items. A 
few states have raised their Praxis I cut scores in 
highly publicized efforts to raise teacher quality. In 
Virginia, for example, prospective teachers must now 
meet a scaled score of 178, or correctly answer about 
71 to 76% of the test items. However, at best, this 
translates into a mediocre performance on a high- 
school level exam. 

Because licensing exams are reported only on a 
pass-fail basis, there is no way of knowing if 
successful candidates score high or just barely make 



the cut. Certainly, bright teacher candidates breeze 
through these tests and many report feeling insulted 
by the tests’ low level. Failing these tests sends a 
clear signal that the candidate is unsuitable for 
teaching. But passing does not tell us whether 
prospective teachers know enough content to teach 
effectively. 

WHO DETERMINES TEST 
CONTENT 

The content in subject-matter licensing exams is 
not, as one might expect, a deliberate, well-considered 
statement of what teachers should know in order to be 
qualified professionals. Licensing examinations are 
meant to establish a floor. In fact, the process used to 
define test specifications and validate items pushes 
content levels to the basement. 

Both NES and ETS are providing their customers, 
the states, with the product they ask for. These 
publishers guarantee that licensing exams are 
psychometrically sound. In addition, the tests have 
undergone a validation process designed to assure that 
they can withstand potential legal challenge of the sort 
recently experienced by Alabama. Such concern has 
led test developers to include only content that they 
can prove a beginning teacher actually uses in his or 
her practice. This practice reduces the likelihood that 
tests will contain content higher than the high school 
level. The minimalist approach to content is justified 
by an assumption that professional growth and 
knowledge will occur over time. However, there is no 
evidence that indeed this happens among all teachers. 

Tests are validated in a multi-phase process. 
Content specifications are provided by groups of 
teachers and representatives of professional 
organizations such as the National Council of 
Teachers of Mathematics, the National Council of 
Teachers of English, the National Science Teachers 
Association, and so on. Because NES contracts 
individually with states, these specifications are 
provided by the customer. 

A bank of items and/or objectives is then 
developed by subject specialists, test experts and 
teachers. The tests are validated by panels comprised 
mostly of novice teachers who have less than five 
years of experience. The panels consider two 
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questions about each item: “Is the knowledge in this 
question used in my teaching?” and “What percentage 
of beginning teachers would answer it correctly?” The 
panel reviews are a fundamental part of the process of 
validating test content. These findings are also 
considered in establishing passing scores, as are 
projected supply and demand of teachers by field and 
impact statements on minority candidates. 

Unless a state suddenly experiences a glut of 
prospective teachers (hardly likely when most are 
claiming shortages) this process cannot accommodate 
raising the bar on licensing exams. In fact, over time 
the process itself creates a downward spiral of 
expectations. 

THE WRONG ANALOGY 

The reason for a minimal approach to teacher 
licensing is at root legal: litigation has established that 
entry-level qualifications must have direct relevance to 
the job. The analogy is with trade: a carpenter cannot 
be required to pass an examination in calculus if 
calculus is never used in framing. Thus, test 
publishers build a firewall against litigation by 
checking with early-career teachers about the 
knowledge they claim to use on the job. 

We believe the trade analogy undermines the 
professional status of teachers. Other professionals- 
including lawyers, accountants, doctors and 
nurses-must pass tests that are notoriously tough. 
Often, as these professionals advance in their fields, 
they must pass still other exams which specifically test 
their increasing content knowledge. Not so in 
teaching. 

Many of those involved with licensing policy 
want to avoid unfairly excluding people from 
becoming teachers. While conducting this study, we 
also heard over and over again that the purpose of 
licensing was to assure that beginning teachers would 
do no harm. But we know from research that poorly 
prepared teachers do harm. And they do the most 
harm to the students who have the least support to fall 
back on for their academic development. 16 

Clearly, states have an interest in preventing 
lengthy suits. Schools of education have an interest in 
showing high success rates among their graduates. 
School adfninistrators, too, have an interest in filling 
vacancies. 



But all of these factors conspire to keep licensing 
criteria minimal. Lost in this process is the students' 
interest in having teachers who have the content 
knowledge needed to help them reach new and higher 
academic standards. 

CAN LICENSING EXAMS MEAN 
SOMETHING? 

Several states are attempting to raise licensing 
requirements. A few, like Massachusetts, are making 
efforts to install higher level tests. Other states, 
notably Maryland and Virginia, are raising passing 
scores. But while the attempts are laudable, raising 
passing scores on low-level examinations will produce 
only modest returns in the long run. What’s needed to 
assure a higher caliber teacher corps is a reevaluation 
of the 

assumptions and 
goals upon 
which the 
current tests are 
based. 

The first 
assumption that 
needs to be 
reexamined is 
that teachers’ 
content 
knowledge 

grows over time. If true, a minimal approach to 
licensing is certainly less risky. Yet there is no 
structure to ensure that a teacher’s intellectual growth 
will happen. Existing career ladders and requirements 
for continuing certification do not emphasize content 
knowledge, on the whole, and they are by no means 
found in every state. States need to build such 
structures. But they also need to certify a solid 
academic foundation in the beginning. 

Second is the assumption that minimum content 
knowledge for K-12 teachers means K-12 content, and 
maybe just a little bit more. Most of the content on 
licensing examinations is most typically found in high . 
school curricula. On the few occasions that tests 
addressed content beyond high school, it was at the 
level of the first or second year of college, never at the 
level of a bachelor’s degree. Such low levels of 
content are insufficient. 



Poorly prepared, 
teachers do harm. 
And they do tike most 
Iwm to tike stMdents 
wlko have tike least 
support to fall badk on 
for their academic 
development 
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The third assumption is about licensure and not 
specifically about the exams. Many states assume that 
passing college courses assures a “college level” 
mastery of content. But there is ample research 
showing that this is not the case. More importantly, 
there is no indication that the content learned in 
college courses is at all relevant to what prospective 
teachers will need to teach. 

In the end, this was the most disturbing aspect to 
us. What we have named knowledge for teaching - 
the deep understanding of key concepts connected to 
K-12 curriculum-is absent from the licensing 
examinations. 

The movement to draft K-12 content standards 
began with a question: What should students know 
and be able to do to be competent, literate high school 
graduates? After long public discussions involving 
educators, subject specialists, industry and civic 
leaders, the answer was content that was significantly 
higher than schools were currently teaching. 

Moreover, it was determined that this higher-level 
content must be mastered by each and every student - 
including those students that schools had traditionally 
left behind. 

This discussion should not be over. It must extend 
to the next logical 
question: What should 
teachers know and be 
able to do to teach their 
students to these new 
standards? When we hold 
such discussions, as some 
college faculty are 
beginning to do now, we 
are bound to find that the 
content requirements for 
teachers also need to be 
significantly higher. And 
licensing examinations 
must reflect this content. 

Raising the level of 
licensing examinations is 
not without risks. The 
threat of litigation will 
continue to loom for 
states and tougher tests 
add to the panic of 
administrators scrambling 



now to put a teacher in every classroom. Even the 
simple act of raising passing scores can have an 
immediate impact on teacher supply. 

Yet the short-term risks of shortages and ill- 
prepared candidates are inconveniences compared to 
the long-term devastation of placing barely qualified 
teachers in charge of our students’ intellectual 
development. 

Teachers truly hold society’s future in their hands. 
There are many gifted teachers currently practicing 
who both know their subjects well and can convey 
that knowledge to their students. But there is no 
present mechanism to ensure that all teachers have 
these qualifications, or will attain them in time. 

Everybody-students, parents, teachers 
themselves, and members of the community-holds a 
high stake in making sure teachers have the 
knowledge they need to teach all students to high 
standards. With public support and political will, 
policymakers and educators can loosen the 
stranglehold that litigation and psychometrics have on 
developing licensing examinations. They can make 
them into instruments that signify high professional 
standards and tests that teachers will be proud to pass. 



Are Teaiclher Camdidlates Adequately 
Prepared? 

A Virginia Example 

Of the 5,000 Virginia teacher candidates who took the PRAXIS I, 
a basic-skills test: 

35% failed the writing portion 
35% failed the math portion 
20% failed the reading portion 

FURTHERMORE, 

Nationwide, nearly half of all teacher candidates would have failed 
to make the Virginia cut. 

Source: Data derived from the Washington Post, February 21, 1998. 

Notes: Passing the reading portion of the exam required that candidates answered 
approximately 71-76% of the questions correctly: passing the math portion of the 
exam required that candidates answered approximately 68-73% of the questions 
correctly: data not available for the writing portion of the exam. 
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ACTING ON THIS INFORMATION 



Some recommendations from the Education Trust 



SHORTTERM ACTIONS FOR STATE, LOCAL AND EDUCATION LEADERS 

All states should assess the academic knowledge of intending teachers, using the most rigorous 
available examinations. 



FOR ELEMENTARY TEACHERS, assessments should measure whether the candidate has at 
least the general knowledge acquired in a four-year liberal arts degree program. None of the 
currently available examinations (with the possible exception of Massachusetts’) does this very 
well, leaving states with two short-term options, neither good. The first is simply to raise the 
passing score on whichever general knowledge exam is currently in use. The second, possible 
only in states that have rigorous, internationally-benchmarked high school exit exams (like New 
York Regents Exams), is to administer that high school exam to intending elementary school 
teachers and demand a “distinguished” or “advanced” performance level. 



FOR SECONDARY TEACHERS, states should require both the essay-rich assessments (for 
example, the Praxis II English Language, Literature and Composition: Essay or Mathematics: 
Proofs, Models and Problems, or NES’ Massachusetts Science Essay Exam) and multiple-choice 
content examinations. This way, both breadth and depth of subject knowledge will be tested. 

Minimum passing scores should be raised. 

School districts should request from all applicants for teaching jobs their scores on relevant 
licensure examinations, as well as copies of college transcripts or other evidence of content 
expertise. While all teacher hiring decisions should factor in this information, high-poverty 
schools, where students are especially dependent upon their teachers for content learning, should 
give test scores and transcripts considerable weight in the hiring process. 



University leaders should note that current state policies do not preclude them from setting 
higher academic standards for graduation from their institutions than states require for licensure. 
Indeed, they would be well advised to consider doing what the Texas A & M System recently 
did: set higher standards for itself than did the State Licensure Board. 



LONGER TERM ACTIONS FOR STATE, LOCAL AND EDUCATION LEADERS 



All states should immediately initiate a process aimed at developing clear academic standards for 
what teachers need to know in the various content areas in order to teach students to the state K- 
12 standards. The standard-setters should start with the K-12 standards, but ask specifically what 
more a teacher needs to know both to have the deep understanding necessary to teach a concept 
well, and also the knowledge necessary to link that concept to others. At the secondary level, 
especially, these standards should represent the kind of knowledge that should be acquired 
during four years of intensive study at the collegiate level. This process should be led by faculty 
from the relevant disciplines, but should also include teachers and education faculty. 
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These new standards for teachers should serve as a framework for rethinking how teachers are 
prepared, including what courses they are required to take. Broad general education 
requirements, which allow students to fulfill science requirements with courses like Astronomy 
or Human Sexuality, may not be sufficient to provide elementary teachers with the content 
background that matches what they will be teaching. The same is true of secondary teachers 
and the academic major: as one group of university-based mathematicians just found, while 
students could learn all of the content necessary to teach to that state’s standards in the courses 
available to a math major, completion of the major would by no means guarantee the right 
combination of content. 



These standards should also guide the development and/or selection of rigorous new academic 
assessments for initial licensure. These should be developed, tested, and put into place as soon 
as possible. 

Passing state licensure exams should not be left to chance. Rather, to ensure that institutions of 
higher education take seriously their responsibility to prepare intending teachers for these 
exams, states should adopt and put into place accountability systems that hold colleges strictly 
accountable for the success of their graduates. These accountability systems should affect the 
arts and science departments that do most of the content preparation of future teachers, as well 
as schools of education. 



Colleges and universities should be held clearly accountable for preparing all of the students in 
their teacher preparation programs — including minority students — to pass state licensure exams. 
As in the state of Texas, institutions that do not succeed with minority students should get help 
improving their success. If they fail to improve, state registration or accreditation should be 
revoked. 



ACTIONS FOR NATIONAL DISCIPLINARY LEADERS 

Like the leaders in the field of mathematics, other disciplinary associations should take the lead 
in designing and carrying out a process for developing model academic standards for teachers at 
each level of the education system. The process should include the relevant subject-matter 
associations from both higher education and K-12. The model standards will serve as a 
reference point for state and local academic leaders as they answer the question “What do our 
teachers need to know to teach children to our standards?” 

The disciplinary associations at both levels should also collaborate on the development of 
rigorous assessments of academic content for prospective teachers. As in the examinations 
developed by the American Chemical Society, these might initially be used voluntarily both by 
colleges (for program improvement purposes) and by individuals (to demonstrate unusual 
mastery). Over time, however, they might be used for initial licensure. 
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Notes 



1 See State tables, pages 20-21 

2 See State tables, pages 20-2 1 

3 Out of the 21 states using ETS secondary English 
language arts examinations, only nine have selected 
the English Language, Literature and Composition: 
Essays. The Mathematics: Proofs, Models and 
Problems exams are used in only seven of the 22 
states administering ETS secondary mathematics 
assessments. See State tables, pps. 20-21. 

4 The ability of Stephen Jay Gould in his book Full 
House (New York: Random House, 1997) to explain 
statistical probabilities rests on a wide range of 
metaphors, from baseball to the sidewalk wanderings 
of a drunk emerging from a bar. 

5 Praxis II, Mathematics: Pedagogy (0065), 1993, p. 10. 
This example comes from the complete tests now 
released publicly. 

6 from TAAG for Praxis I, Princeton NJ: ETS, 1998, 
p. 57 number 3 

7 from the Massachusetts Teacher Tests™ Test 
Information booklet, Massachusetts Department of 
Education, 1998. p. 42 



8 TAAG for Biology and General Science, 1998, 
Biology: Content Knowledge, Part 2 (0232), 
Princeton NJ: ETS, 1998, p. 24 

9 The Praxis Series, NTE Core Battery Tests, Practice 
and Review, Princeton NJ: ETS, 1992, p. 49 

10 TAAG for Praxis I, Princeton NJ: ETS, 1998, p. 43, 
number 4 

1 1 Interestingly, the computer version of Praxis I had 
more purposeful prompts, some of which required 
the defense of an argument, although they still 
lacked a specific audience. 

12 TAAG Biology and General Science, Princeton NJ: 
ETS. 1998, p. 69 

13 Praxis II, TAAG, English Language, Literature, and 
Composition: Content Knowledge (0041) 

14 The Praxis II scaled scores are unweighted, making 
a straight percentage of correct items a relevant 
indicator. 

15 Richardson v. Lamar County (AL) Bd. of Educ. 
1989 

16 see Haycock, Kati, “Good Teaching Matters,” 
Thinking K-16, Summer 1998, Washington DC: 
Education Trust. 
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How to Mead TMs Clhairt 

The chart below allows the reader to identify which states are using which content knowledge examinations for teacher 
licensure in elementary education, secondary English/language arts, and secondary mathematics. Actual requirements can 
vary (some states, for example, require tests for out-of-state applicants only): readers should contact the appropriate state 
agency with questions about requirements. 

There are two major test publishers: Educational Testing Service (ETS) which publishes the Core Battery, NTE, and Praxis 
series, and National Evaluation Systems (NES) which designs state-specific examinations. Tests listed on the chart include: 

ETS : PRAXIS I: a three-part exam in Mathematics, Reading and Writing 
PRAXIS II: subject area tests 

Core Battery: consists of three tests: general knowledge professional knowledge and communication skills. 

NTE: subject area tests 
NES: state-specific exams 

OTHER: “**’ denotes that licensing exams are either developed by the state or administered by the Local 
Education Authority, the State Education Authority, or an Institution of Higher Education. 

The chart shows, reading from left to right across rows, by state: 

° The basic skills exam used for Elementary teaching. 

° The basic skills exam used for Secondary teaching. 

° The English subject area exam(s) used for Secondary teaching. 

° The Math subject area exam(s) used for Secondary teaching. 

Please keep in mind: 

° We are listing SUBJECT AREA exams only; not oedagogy exams. 

° Tests being pnased out appear in italics. Both the National Teachers Exam (NTE) and the Core Battery will be 

completely phased out by June 2000, and replaced with PRAXIS II. They are included in the chart because they 
are in use for the 1998-99 year covered by tne chart. 

° We limited our study to English/language arts, mathematics and science. The chart includes only 

English/language arts and mathematics subject area exams for secondary teaching. States may require additional 
subject area exams in other subject areas. 

° Because NES contracts with individual states, access to their tests is in the hands of the appropriate state agency. 

We had access to complete examinations from a single NES state, which we are prohibited from naming, and study guides 
from six others. Due to these limitations, NES exams are listed simply as NES. 

The data used to build this chart are drawn from Under standingYour Praxis Scores , The Praxis Series: Professional 
Assessments for Beginning Teachers, 1998-99 Spring Edition , (Educational Testing Service), and the NASDTEC Manual on 
the Preparation and Certification of Educational Personnel, 1998-99 (National Association of State Directors of Teacher 
Education & Certification). 



Conutenut Knowledge Examms, By State 1998-99 

ELEMENTARY SECONDARY 



State [ 

Alabama | 


I All Elementary f 


| All Secondary 
* 


Secondary English 


Secondary Math 


[Alaska 


PRAXIS I 




PRAXIS I 






Arizona 


* 


. 


[Arkansas 


PRAXIS I 




PRAXIS I 


PRAXIS II: E L L & C (essays) 


PRAXIS II: Math (content) 


1^ 


1 


PRAXIS II: E L L & C (content) 


PRAXIS R: Math (proofs I & H) 


[California 


NES 




NES 


NES 


NES 1 


1 




II 


PRAXIS II: E L L & C (essays) 


PRAXIS R: Math (proofs I & H) 


[Colorado 


NES 




NES 


NES 


NES 1 


Connecticut 


PRAXIS I 


n 


| PRAXIS I 


PRAXIS II: E L L & C (essays) 


PRAXIS II: Math (content) 


1 PRAXIS IT: E L L & C (content! 1 


Delaware 


PRAXIS I 


PRAXIS I 


[DC 


PRAXIS I 




PRAXIS I 


PRAXIS II: E L L & C (content) 


PRAXIS II: Math (content) [ 






ii 




PRAXIS II: Math (proofs I) 


[Florida 


PRAXIS I 




PRAXIS I 


PRAXIS II: E L L & C (content) 


NTE: Math I 


Georgia | 


| PRAXIS I 




PRAXIS I 


PRAXIS n:ELL&C (essays) 


PRAXIS II: Math (content) 


i 








PRAXIS II: E L L & C (content) 


PRAXIS 0: Math (proofs I) ! 


Hawaii 


1 PRAXIS I 


|| PRAXIS I 


PRAXIS II: E LL & C (content) 


PRAXIS II: Math (content) 


[Idaho 1 
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(Content Knowledge Exams, By State 1998-99 





ELEMENTARY 




SECONDARY 






State 


1 All Elementary | All Secondary 


Secondary English 


Secondary Math 




| Illinois 


NES 


NES 


NES 


NES 




Indiana 


| Core Battery 


Core Battery 


NTE: EL&L 


NTE: Math 




|lowa ] 


Kansas 


PRAXIS I 


PRAXIS! 


[Kentucky 


PRAXIS! 


PRAXIS I 


PRAXIS II: E L L & C (essays) 


PRAXIS H: Math (Content) 










PRAXIS II: E L L & C (content) 


PRAXIS II: Math (Proofs I) 




l 


Louisiana 


| Core Battery 


Core Battery 


NTE: EL&L 


NTE: Math 




[Maine 


PRAXIS VCB 


PRAXIS VCB 








Maryland 


( Core Battery || Core Battery 


NTE: EL&L 


NTE: Math 




[Massachusetts 


NES 


NES 


NES 


NES 




Michigan 


NES 


NES 


NES 


NES 




[Minnesota 


PRAXIS I 


PRAXIS I 








Mississippi 


PRAXIS I 


| PRAXIS I 


NTE: EL&L 


NTE: Math 




[Missouri 


* 


* 


PRAXIS H: E LL&C (content) 


PRAXIS II: Math (content) 




Montana 


PRAXIS! 


PRAXIS! 


(Nebraska 


PRAXIS! 


PRAXIS I 








Nevada 


PRAXIS! 


| PRAXIS I 


PRAXIS Hi E L L & C (essays) 


PRAXIS II: Math (content) 




| PRAXIS II: Math (proofs I) j 


New Hampshire 


| PRAXIS! 


PRAXIS! 


[New Jersey 




* 


PRAXIS H: E L L & C (content) 


PRAXIS II: Math (content) 




New Mexico || Core Battery || Core Battery 


[New York 


NES ICB 


NES/CZ? 


NES 


NES 




North Carolina | 


1 PRAXIS! || PRAXIS! 


PRAXIS II: E L L & C (essays) 


PRAXIS II: Math (content) 




1 PRAXIS II: ELL&C (content! j 


North Dakota 




iOhio 


Core Batterv 


Core Battery 




NTE: Math 




Oklahoma 


PRAXIS I/NES I 


I PRAXIS I/NES 


NES 


NES 




lOre&on 


PRAXIS I/NES 


PRAXIS I/NES 


PRAXIS 0: E L L & C (essavs) 


PRAXIS II: Math (content) 










PRAXIS II: E L L & C (content) 


PRAXIS II: Math (Proofs I & II) 




[Pennsylvania 


Core Battery 


Core Battery 


PRAXIS II: E L L & C (content) 


PRAXIS II: Math (content) 




Rhode Island 


Core Battery 


Core Battery 








[South Carolina 


PRAXIS I 


PRAXIS! 


NTE: EL&L 


NTE: Math 




South Dakota 






[Tennessee 


PRAXIS I 


PRAXIS I 


PRAXIS II: E L L & C (essays) 


PRAXIS II: Math (content) 




r 1 


1 


i 


PRAXIS H: E L L & C (content) 






[Texas 


PRAXIS I/NES 


PRAXIS I/NES 


NES 


NES 




Utah 






[Vermont [ 


Virginia ( 


| PRAXIS! 


PRAXIS! 


NTE: EL&L 


NTE: Math 




(Washington j 


West Virginia [ 


! PRAXIS I/NES || PRAXIS I/NES 


PRAXIS II: E L L & C (content) 


PRAXIS II: Math (content) 




‘Wisconsin 


PRAXIS I 


PRAXIS I 








Wyoming | 


1 * 1 


1 * 









Test Title Key: EL&L — “English Language & Literature (0040) ELL&C (essays) = “English Language ; Literature & Composition: Essays 
(0042) ELL&C (content) = “English Language, Literature & Composition: Content Knowledge (0041) Math (Proofs I) = “Mathematics: 
Proofs, Models & Problems, Part 1 (0063) Math (Proofs II) = “Mathematics: Proofs, Models & Problems, Part II (0064) 
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STATE TABLES 

(Comiteinit Knowledge Exams: Passing Scoires hy State 1998-99 



Praxis II — Eng Lang, Lit & Comp: Content (0041) 



State 


Passing 

Score 


Estimated % 
Correct To 
Pass 


% of Test 
Takers 
Nationally 
who would 
fail to make 
this cut 


Connecticut 


: 172 76-79% 


34 


Florida 


165 68-71% 


21 


Hawaii 


164 68-71% 20 


Oregon 


164 68-71% 


20 


Ceorgia 


163 68-71% 18 


Arkansas 


| 159* 


64-67% 


12 


Missoun 158 64-67% ; 11 


Tennessee 1 57 64-67% 


10 


New Jersey 1 55 ; 64-67% 


8 


West Virginia 155 64-67% 


1 8 


North Carolina 154(9-12) 59-63% 


7 


152 (6-8) 59-63% 


6 


Pennsylvania >153 59-63% 


7 


DC 142 


51-55% 


2 


Kentucky 138 54-55% 


2 


l t 

‘Effective July 1999 





Praxis II — Eng, Lang, Lit & Comp: 
Essays (0042) 



State 


Passing 

Score 


% of Test 
Takers 
Nationally 
who would 
fail to make 
this cut 


California 


160 


30 


Connecticut 


160 


30 


Nevada 


155 


21 


Arkansas 


150“ 


14 


Oregon 


145 ‘ 


8 


Georgia 


135 


4 


Kentucky 


135 


4 


North Carolina 


135 


4 


Tennessee 


* 


L. - 


‘Test Required - Passing score not set 


“ Effective July 1999 



Praxis II — Mathematics: Content (0061) 



State 


Passing 

Score 


Estimated % 
Correct To 
Pass 


% of Test 
Takers 
Nationally 
who would 
fail to make 
this cut 


Oreqon 


I 147 


64-66% 


70 


Connecticut 


141 


60% 


60 


DC 


141 


60% 


60 


Kentucky 


141 


1 60% 


60 


Missouri 


137 


56-58% 


56 


Arkansas 


136“ 


56% 


51 


Hawaii 


136 


56% 


51 


Tennessee 


136 


56% 


51 


North Carolina 


133 


52-54% 


48 1 


West Virginia 


133 


52-54% 


48 


New Jersey 


130 


50-52% 


43 


Pennsylvania 


127 


48-50% 


38 


Georqia 


124 


46% 


34 


Nevada 


* 






* 


Multiple Scores Required 
“ Effective July 1999 



Praxis II — Math: Proofs, Models & 
Problems Part I (0063) and Part II (0064) 



State 


Passing 

Score 


% of Test 
Takers 
Nationally 
who would 
fail to make 
this cut 


Part 1 


California 


170 


59 


DC 


154 


37 


Nevada 


152 


35 


Arkansas 


144* 


26 


Oregon 


144 


26 


Kentucky 


141 


23 


Georgia 


139 


20 


Part II 


California 


159 


72 


Oreqon 


140 


49 



* Effective July 1999 
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STATE TABLES 



CoMteMt Knowledge Exams: Passing Scores by State 1998-99 



Praxis I — Writing (0720) 



State 


Passing 

Score 


% of Test 
Takers 
Nationally 
who would 
fail to make 
this cut 


Virginia 


176 


; ' 56 ' 


Wisconsin 


174 


32 


Alaska 


174 


1 32 


Texas 


173 


23 


Arkansas 


173* 


23 


South Carolina 


173 


23 


North Carolina 


173 


23 


Delaware 


173 


23 


Tennessee 


172 


15 


Oklahoma 


172 


15 


New Hampshire 


172 


15 


Nevada 


172 


15 


Nebraska 


172 


15 1 


Mississippi 


172 


15 


Minnesota 


172 


15 


Kentucky 


172 


15 


Kansas 


172 


15 


Georgia 


172 


15 


West Virginia 


171 


11 


Oregon 


171 


11 


Hawaii 


171 


11 


Florida 


171 


11 


DC 


171 


11 


Montana 


170 


7 


Maine 


168 


3 ! 


* Effective July 1999 



NTE — English Language & 
Literature (0040) 



State 


Passing 

Score 


% of Test 
Takers 
Nationally 
who would 
fail to make 
this cut 


Louisiana 


550 20 


Mississippi 


530 14 


Virginia 


520 11 


South Carolina 


500 


7 


Indiana 


500 7 


Maryland 


500 7 


Arkansas 


490 6 





NTE — Mathematics (0060) 



State 


Passing 

Score 


% of Test 
Takers 
Nationally 
who would 
fail to make 
this cut 


Virginia 


580 


34 


South Carolina 


560 


25 


Louisiana 


550 


20 


Indiana 


530 


13 


North Carolina 


530 


13 


Ohio 


530 


13 


Mississippi 


520 


11 


Maryland 


520 


11 


Kentucky 


500 


7 


Arkansas 


* 




Florida 


* 




* Multiple scores required. 



The data used to build these tables are drawn from the following documents: 



Passing Score: Understanding Your Praxis Scores, The Praxis Series: Professional Assessments for Beginning Teachers, 
1998-99 Spring Edition, (Educational Testing Service, 1999). 



Estimated % Correct to Pass: Test Analysis: Pre-Professional Skills Tests, October 28, 1989 Administration, Form 
3MPS1, Unpublished Report, (Educational Testing Service, 1990). 



% of Test Takers Nationally Who Would Fail to Make this Cut: The Praxis Series, Professional Assessments for 
Beginning Teachers: 1995-96 Percentile Ranks and Summary Statistics, (Educational Testing Service, 1996). 
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